Vision-based interactive projection system

ABSTRACT

Techniques are provided to improve interaction between a user and a projection system. In some embodiments, an image of a user in front of a display screen is captured. An inference can then be made as to whether a user is touching a display screen based on an analysis of shadows and/or variation of brightness (i.e., intensities) across pixels in the image. For example, it may be inferred that the object is: (1) approaching the screen when a region surrounding a top of the object is characterized by a relatively small brightness variation; (2) hovering near the screen when the brightness variation is large and the region includes a dark extremum (caused by a shadow); and (3) touching the screen when the brightness variation is large and the region includes a light extremum.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/645,390, entitled “VISION-BASED INTERACTIVE PROJECTION SYSTEM”, filedOct. 4, 2012, which claims the benefit and priority of U.S. ProvisionalApplication No. 61/544,983, filed on Oct. 7, 2011, which is herebyincorporated by reference in its entirety for all purposes.

BACKGROUND

Projectors are widely used in schools, trade shows, businesses, museums,and anywhere that a screen is needed to be shown on a large surface. Aprojector may be coupled to a computer, and may then project computerdisplays onto a screen. The user is frequently standing near the screen,and not near the computer. Thus, it is difficult to coordinate thecomputer's operation. An external device may be used, e.g., to progressthrough slides in a presentation. However, external devices arefrequently confusing and of limited utility. It would be advantageousfor a system to allow a user to have increased interaction with aprojection system.

SUMMARY

Techniques are provided to improve interaction between a user and aprojection system. In some embodiments, an image of a user in front of adisplay screen is captured. An inference can then be made as to whethera user is touching a display screen based on an analysis of shadowsand/or variation of brightness (i.e., intensities) across pixels in theimage. For example, it may be inferred that the object is: (1)approaching the screen when a region surrounding a top of the object ischaracterized by a relatively small brightness variation; (2) hoveringover the screen when the brightness variation is large and the regionincludes a dark extremum (caused by a shadow); and (3) touching thescreen when the brightness variation is large and the region includes alight extremum.

Lights can be appropriately arranged and configured (e.g., in aprojection system) to ensure that the brightness variations areappropriately associated with desired states characterizing the user'sinteraction with the screen. For example, a projection system caninclude: projection optics, a front light, a back light and a camera—allembedded in a shared housing. The front light can be configured toilluminate a region in front of the display screen, such that an object(e.g., a fingertip) hovering over the screen is surrounded bysubstantially no shadow in an image captured during the illumination bythe camera. The back light can be configured to illuminate the displayscreen, such that the object hovering over the screen is surrounded by ashadow in an image captured during the illumination by the camera. Thecamera can be configured to image the screen during the front-lightillumination (to produce a first image) and to further image the screenduring the back-light illumination (to produce a second image). Thefirst image can be used to estimate a location of the object, and thesecond image can be used to infer a state of an object (e.g., bycharacterizing brightness variation across pixels in a region and/oridentification of whether an extremum in the region was bright or dark).

In some embodiments, a method is provided. A display can be projectedonto a surface. A region near the surface can be illuminated. An imageof the surface can be captured during the illumination. A variation ofintensities across pixels in at least a portion of the image can beanalyzed. A polarity associated with the portion of the image can bedetermined. A determination can be made as to whether an object istouching the surface based at least partly on the analysis of thevariation and on the determined polarity.

In some embodiments, a projector system is provided. Projection opticscan be configured to project a display on a surface. A light source canbe configured to emit light in a direction towards the surface. A cameracan be configured to capture an image of the surface during the emissionof light. An image analyzer can be configured to detect a variation ofintensities across pixels in at least a portion of the image and todetermine a polarity associated with the portion of the image. A statedetector can be configured to determine whether an object is touchingthe surface based at least partly on the variation detection and thedetermined polarity.

In some embodiments, a projector system is provided. The projectorsystem can include means for projecting a display onto a surface andmeans for illuminating a region near the surface. The projector systemcan also include means for capturing an image of the surface during theillumination and means for analyzing a variation of intensities acrosspixels in at least a portion of the image. The projector system canfurther include means for determining a polarity associated with theportion of the image and means for determining whether an object istouching the surface based at least partly on the analysis and on thedetermined polarity.

In some embodiments, a non-transitory computer-readable medium isprovided. The medium can contain instructions which, when executed by aprocessor, can cause the processor to access an image of a surface onwhich a display is projected and analyze a variation of intensitiesacross pixels in at least a portion of the image. The instructionswhich, when executed by the processor, can further cause the processorto determine a polarity associated with the portion of the image anddetermine whether an object is touching the surface based at leastpartly on the analysis of the variation and on the determined polarity.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an embodiment of a projection system 100.

FIG. 2 shows example representations of a timing circuit's inputs andoutputs.

FIG. 3 shows a schematic of one embodiment of a timing circuit.

FIG. 4 shows an embodiment of a method for inferring whether an objectis touching display screen.

FIG. 5 shows an embodiment of a method for estimating a location of anobject.

FIGS. 6a-6b show embodiments of methods for determining whether anobject is touching the screen.

FIG. 7 shows an example of a state machine that can track changes insituations when an object approaches the screen.

FIG. 8 shows an example of a projection system.

FIG. 9 provides a schematic illustration of one embodiment of a computersystem.

FIGS. 10a-10c show images associated with an example situation in whicha hand is hovering over a screen while a back light source isilluminating the scene.

FIGS. 11a-11c show images associated with an example situation in whicha finger is touching a screen while a back light source is illuminatingthe scene.

DETAILED DESCRIPTION

Techniques are provided to improve interaction between a user and aprojection system. In some embodiments, an image of a user in front of adisplay screen is captured. An inference can then be made as to whethera user is touching a display screen based on an analysis of shadowsand/or variation of brightness (i.e., intensities) across pixels in theimage. For example, it may be inferred that the object is: (1)approaching the screen when a region surrounding a top of the object ischaracterized by a relatively small brightness variation; (2) hoveringover the screen when the brightness variation is large and the regionincludes a dark extremum (caused by a shadow); and (3) touching thescreen when the brightness variation is large and the region includes alight extremum.

Lights can be appropriately arranged and configured (e.g., in aprojection system) to ensure that the brightness variations areappropriately associated with desired states characterizing the user'sinteraction with the screen. For example, a projection system caninclude: projection optics, a front light, a back light and a camera—allembedded in a shared housing. The front light can be configured toilluminate a region in front of the display screen, such that an object(e.g., a fingertip) hovering over the screen is surrounded bysubstantially no shadow in an image captured during the illumination bythe camera. The back light can be configured to illuminate the displayscreen, such that the object hovering over the screen is surrounded by ashadow in an image captured during the illumination by the camera. Thecamera can be configured to image the screen during the front-lightillumination (to produce a first image) and to further image the screenduring the back-light illumination (to produce a second image). Thefirst image can be used to estimate a location of the object, and thesecond image can be used to infer a state of an object (e.g., bycharacterizing brightness variation across pixels in a region and/oridentification of whether an extremum in the region was bright or dark).

FIG. 1 shows an embodiment of a projection system 100. The projectionsystem may include a projector 105. The projector may be configured toproject a reference image to a screen, thereby creating a projectedimage, e.g., projected on a display screen 160. Projector 105 may alsobe configured to project an image generated in association with anapplication or program or other aspect of operation of a computer.Display screen 160 may include, e.g., a white board, a pull-down screen,a wall, etc. Display screen 160 may be provided in a set with projectionsystem 100—either being independent and separable to system 100 or beingconnected to system 100. Display screen 160 may have a substantiallyrectangular shape and may have a diagonal dimension of at least about,about, or less than about, 1 mm, 1 cm, 10 cm, 100 cm, 1 m, 3 m, 5 m, or10 m. Display screen 160 may be comprised of a material that is at leastpartly reflective. In some instances, display screen 160 has areflectance of at least about 1% or 10% and/or less than about 20% or40%. Although the term “display screen” is used herein, the displayscreen is not restricted to elements or components manufactured with theintent that images be displayed thereon. A display screen as used hereinmay comprise any surface on which light and/or an image may beprojected, such as a wall, a floor, a piece of paper, a window, or anyother surface. In some embodiments, a surface may be created by steam orcondensation. In some embodiments, the surface comprises a user's hand,clothing, or other element associated with the user.

Projector 105 may include, e.g., a short-throw projector. Projector 105may have a throw ratio (a distance from a projector to a screen dividedby a diagonal distance of a projected image) of less than about 1 orless than 0.5. Projector 105 may include projector optics, such as aprojector light source 105 a and mirror 105 b. In some embodiments,projector 105 is an independent device within projection system 100and/or is housed within a single proctor housing. In other embodiments,projector 105 represents collection of projector optics and/or otherprojector components that are not separately housed from the rest ofprojection system 100.

Projection system 100 may include at least one front light source 100and/or at least one back light source 115, each configured to emitlight. The light emitted by one or both of the light sources may benon-visible. In some embodiments, the light emitted by one or both ofthe light sources comprises or consists of infrared light. One or bothof the light sources may comprise one, more or an array of componentlight sources, such as light-emitting diodes (LED). In one embodiment,back light source 115 comprises a single light source, and front lightsource 110 comprises an array of component light sources (e.g., LEDs).In some embodiments, a light source comprises a plurality of componentlight sources positioned with a density such that light emitted from thecollection of component light sources appears as a single light source.

Front light source 100 may be positioned in front of projector 105and/or projection optics 105 a and 105 b. Front light source 100 may beattached to a front side of projector 105 and/or to a front portion of abottom side of projector 105. Front light source 100 may be positionedbetween projector 105 and display screen 160 and/or between back lightsource 115 and a display screen 160. In some embodiments, front lightsource 100 is positioned near a front end of projector 105. In someinstances, a horizontal separation between display screen 160 and frontlight source 100 is less than about 1.5 meters.

Front light source 100 may be configured to emit light from projectionsystem 100 in a first direction. In some embodiments, light is initiallyemitted from front light source 100 in the first direction. In someembodiments, optics are used to direct the initial light to propagate inthe first direction. The first direction may comprise a firstx-component (e.g., in a direction parallel to display screen 160 and toceiling 155) and a first y-component. (e.g., in a direction orthogonalto a ceiling 155, floor or other surface that supports system 100)

Front light source 100 and/or optics associated with the light sourcemay be configured such that light emanating from the source and exitingthe projection system illuminates an object (e.g., a finger, a hand, orother object, such as a wand, a pointing device, etc.) touching displayscreen 160. Thus, it may be configured to illuminate a region directlyin front of display screen 160. Proper configuration to properly and/orfully illuminate an object of interest may involve appropriatelysetting, e.g., a position, orientation and/or brightness of light source100 and/or selection and placement of optics directing light emittedfrom the light source 110. In some instances, an object touching displayscreen 160 may reflect some light sourced by front light source 100 thatis mirrored on the screen and seen by camera 120. As explained below,this is frequently undesirable in these circumstances. Therefore,brightness, position and orientation of front light source 100 and anyassociated optics may be configured to reduce or minimize this mirroreffect.

Back light source 115 may be positioned near a back of projector 105.Back light source 115 may be positioned behind front light source 100and/or projection optics 105 a and 105 b. Back light source 115 may beattached to a back side of projector 105 and/or to a back portion of abottom side of projector 105.

Back light source 115 may be configured to emit light from projectionsystem 100 in a second direction. In some embodiments, light isinitially emitted from back light source 115 in the second direction. Insome embodiments, optics are used to direct the initial light topropagate in the second direction. The second direction may comprise asecond x-component and a second y-component. The ratio of the secondx-component to the second y-component may be larger than a ratio of thefirst x-component to the first y-component. The ratio of the secondx-component to the second y-component may be at least about 0.25, 0.5,1, 2 or 5. The horizontal separation between front light source 100 andback light source 115 may be at least about 0.1, 0.25, 0.5, 1 or 2 feet.In one embodiment, light emitted from back light source 115 can beprojected through a lens of projector 105. For example, a prism may bepositioned along an optical path between back light source 115 and alens of projector 105.

In some instances, an object touching display screen 160 may reflectsome light sourced by back light source 115 that is mirrored on screen160 and seen by camera 120. As explained below, this is frequentlydesirable in these circumstances. Thus, brightness, position andorientation of back light source 100 and any associated optics may beconfigured to produce this mirror effect.

Projection system 100 may include a camera 120. Camera 120 may beattached to a bottom of projector 105. Camera 120 may be positioned neara front of projector 105 (e.g., in front of projection optics 105 a and105 b). Camera 120 may be positioned between front light source 100 andback light source 115. Camera 120 may be oriented to capture images of afront surface of display screen 160. Camera 120 may comprise a filter,such as an infrared filter, such that it can image display screen 160.In some embodiments, in addition to or instead of camera 120, system 100includes another type of light sensor such as an IR sensor.

In some instances, an object touching display screen 160 may reflectsome light sourced by a light source (110 or 115) that is mirrored onthe screen and seen by camera 120. This mirror effect may be undesirablein some circumstances (e.g., when light is emitted from front lightsource 110) and desirable in other circumstances (e.g., when light isemitted from back light source 115). Camera 120 may be configured (e.g.,by setting image-capturing parameters, such as shutter and gain) tominimize, reduce, maintain, enhance or maximize this mirror effect. Insome instances, camera configurations (e.g., parameters) changedepending on which light source is emitting light (e.g., to reduce theeffect when front light source 100 is emitting light and to enhance theeffect when back light source 115 is emitting light). In some instances,camera configurations remain constant across conditions, and, e.g.,other strategies, such as configuring a light source's position, powerand/or orientation may achieve the desired state-specific mirror effect.

Projection system 100 may include a timing circuit 125. Timing circuit125 may be coupled to one or both of front light source 100 and backlight source 115. Timing circuit 125 may control when one or both lightsources emit light, e.g., by sending on/off signals, which may controlpower supplied to each light source. Timing circuit 125 may receiveinput from camera 120. Thus, for example, timing circuit 125 may beinformed when a camera is beginning to or is about to begin imagingdisplay screen 160. Timing circuit 125 may be configured to ensure thatonly one of light sources 110 and 115 is emitting light at a time and/orsuch that at least one light source 110 and/or 115 is emitting lightduring a time interval (i.e., frame) during which camera 120 is imagingdisplay screen 160.

In some instances, timing circuit 125 sends alternating on/off signalsto the light sources 110 and 115. For example, upon receiving a signalthat a camera is imaging a first frame, timing circuit 125 may send anon signal to front light source 110. Upon receiving a signal that thefirst imaging frame is complete, timing circuit may send an off signalto front light source. Similar signals may be sent at similar times toback light source 115 in association with a second imaging frame. FIG. 2shows signals representing this embodiment. The top trace represents acamera signal received by timing circuit 125 from camera 120. The middletrace represents a first output of the timing circuit sent to controlfront light source's light emission. The bottom trace represents asecond output of the timing circuit sent to control back light source'slight emission. Thus, in this embodiment, light is emitted from one—butonly one—of light sources 110 and 115 during each camera frame.

Timing circuit 125 may include any of a variety of timing circuits.Timing circuit 125 may include, e.g., one or more integrated circuits,resistors, transistors, and/or diodes. Timing circuit 125 may receiveone or more inputs from, e.g., camera 120, a clock, and/or a processor.Timing circuit 125 may include one, two or more outputs. FIG. 3 shows aschematic of one embodiment of timing circuit 125.

Projection system 100 may include a housing 130. At least part or mostof housing 130 may include a metal or plastic material. Housing 130 mayat least partly or fully surround one, more or all of: projector 105,front light source 110, back light source 115, camera 120 and timingcircuit 125. In some instances, housing 130 at least partly or fullysurrounds all non-housing parts of projection system 100. In someembodiments, housing 130 includes one or more lenses (or openings),e.g., in a bottom and/or front surfaces. These lenses may allow theprojection system 100 to emit light from front light source 110, backlight source 115, and/or projector light source 105 a. A lens mayfurther allow camera 120 to collect external images (e.g., at or infront of display screen 160). In some embodiments, housing 130 comprisesone or more surfaces that are at least partly transparent. In someembodiments, one, more or all of: projector 105, front light source 110,back light source 115, camera 120 and timing circuit 125 may be directlyor indirectly mechanically coupled together in a configuration that doesnot include a housing. For example, all of these elements may be securedto an open frame. The term housing is used to describe embodiments thatare not only enclosed or partially enclosed, but also to describeembodiments in which elements are openly or loosely mechanicallycoupled.

As described in more detail below, projection system 100 may alsoinclude, e.g., a storage device, a processor, and/or a computer. In someembodiments, projection system 100 includes a transmitter to transmitdata (e.g., from camera 120) to an external storage device, processorand/or computer. Projection system 100 may also include a receiver toreceive data (e.g., from a clock, external computer, or auser-controlled device). One, more or all of: the storage device,processor, computer, transmitter, and receiver may be partially or fullyenclosed in the housing discussed above or otherwise mechanicallycoupled to one or all of the elements discussed above.

System 100 or a portion thereof may comprise a self-contained unit thatmay be portable or may otherwise be configured as an integrated unit,for example, when enclosed by the housing discussed above. System 100may be used to establish a touch screen on virtually any surface in someembodiments. Thus, the system 100 may be carried to a variety of or anylocation and used to establish a touch-based computer system, forexample, with respect to a wall in a conference room or on a table. Insome aspects, system 100 is integrated into a mobile device such as asmartphone, and may be configured as a pico projector. In some aspects,front light source 100 and back light source 115 may be disposed side byside or one on top of the other in addition to or instead of beingaligned towards the front and back of system 100.

FIG. 4 shows an embodiment of a method 400 for inferring whether anobject is touching display screen 160. At 405, a reference image (or, inother words, a display) is projected (e.g., by projector 105) ontodisplay screen 160. The reference image may be, e.g., a digital imagestored on and/or transmitted by a computer coupled to projection system100. The reference image may include, e.g., a presentation slide, adocument, a desktop, a user interface, a picture, an animation, etc.Whether process 400 proceeds to block 410 or to block 430 depends on acurrent time step. In some instances, process 400 initially alwaysproceeds to one of blocks 410 and 425 (e.g., always performing block 410before block 430). In some instances, process 400 depends on an absolutetime.

At 410, a first region near display screen 160 is illuminated (e.g., byfront light source 110). The first region may comprise or consist of aregion just in front of display screen 160. For example, theillumination may be configured such that a user's finger will likely beilluminated if the user, the user's hand and/or the user's finger ispositioned between projection system 100 and display screen 160 andtouches display screen 160. In some embodiments, a direction andbrightness of the illumination may be configured such that, in an imagecaptured at 415, there is minimal or no shadow surrounding an object inthe first region. The illumination may comprise illumination bynon-visible light. This first-region illumination may be in addition tolight emitted to project the reference image in 400 (which would includeemission of visible light).

At 415, a first image is captured of the first region. The first imagemay be captured by a single camera (e.g., camera 120) or a set ofcameras. In some instances, the capturing cameras are at least partlyenclosed within a shared outer housing of a projection system 100. Thefirst image may be captured while the first region is being illuminated.In some instances, for example, a shutter of camera 120 is open during atime period substantially the same as the time period during which thefirst region is being illuminated. Opening the shutter for a longer timeperiod may harm image quality, and opening the shutter for a shortertime period may waste power required to power a light sourceilluminating the first region. The captured image may include atwo-dimensional digital image. The captured image may include part orall of display screen 160.

At 420, an estimation is made as to a location of an object of interest.The location may comprise location or a top of the object, a top of ashadow of the object, or a bright area above the object (produced basedon contact between the object and the screen). The location may comprisea two-dimensional location, comprising an x- and y-coordinate. The x-and y-axes may be axes associated with the second image or axesassociated with display screen 160. In one instance, the locationcomprises coordinates that would accurately reflect an object's locationonly if the object were touching display screen 160. In some instances,the location comprises a three-dimensional location. For example, thelocation may be estimated based on a set of first images capturedsimultaneously, or a computer-vision technique may be used to estimate az-coordinate of an image.

The estimation may be made based at least partly on the first image,such that an object in the first image may be surrounded by little or noshadow. Thus, it may be relatively easy to identify, e.g., a tip of afinger, even regardless as to the object's proximity or contact withdisplay screen 160 in some embodiments. Location-identifying techniquesmay include, e.g., a computer-vision technique (e.g., feature detection,edge detection, etc.). For example, edges of an object may be used toidentify an object's location.

If other analysis has been performed to allow assessment as to whetherthe object is touching the screen, process 400 may continue to 425, atwhich interaction between the projected image and the object is allowed.The interaction can depend on the estimated location (e.g., allowing auser to select or move icons projected at the estimated location) incombination with the determination of the whether the object is touchingthe screen, or on only one of the estimated location or touchdetermination. Following the allowed interaction or if an assessment asto whether the object is touching the screen cannot yet be made, process400 returns to block 405, at which the reference image continues to beprojected or is refreshed. In some embodiments, a refresh rate of thereference image is independent of the method 400.

Process 400 can then continue to block 430, at which a second regionnear display screen 160 is illuminated (e.g., by back light source 115).The second region may comprise or consist of a front surface of displayscreen 160 and/or a region just in front of display screen. For example,the illumination may be configured such that a user's hand will likelybe illuminated if he stands between projection system 100 and displayscreen 160 and touches display screen 160 or hovers his finger close tothe screen 160. In some embodiments, a direction and brightness of theillumination may be configured such that, in an image captured at 435,there is a shadow or a predominate shadow surrounding an object in thesecond region if the object is not touching display screen 160. Theillumination may comprise illumination by non-visible light. Thissecond-region illumination may be in addition to light emitted toproject the reference image in 400 (which would include emission ofvisible light).

As shown, the first region and second regions can be illuminated duringdifferent, and possibly, non-overlapping time periods. Time circuit 125may control the time periods during which one or both regions areilluminated. The first and second regions may also comprise differentspatial regions, e.g., based on different locations of light sourcesilluminating the region. Nevertheless, the first and second regions mayoverlap (e.g., both may illuminate all or part of display screen 160when no user is in front of the screen).

At 435, a second image is captured of the second region. The secondimage may be captured by a single camera (e.g., camera 120) or a set ofcameras. In some instances, the capturing camera/s are at least partlyenclosed within a shared outer housing of a projection system 100. Thefirst and second images may be captured by the same camera (e.g., asingle camera inside projection system 100) or set of cameras. The firstand second images may be from a same perspective. The second image maybe captured while the second region is being illuminated. In someinstances, for example, a shutter of camera 120 is open during a timeperiod substantially the same as the time period during which the secondregion is being illuminated. The captured image may include atwo-dimensional digital image. The captured image may include part orall of display screen 160.

The first and second images may be of a substantially similar or thesame perspective. In one embodiment, a single imaging device capturesboth images in a substantially similar or identical manner, the onlydifference being the time at which the images were captured. In someembodiments, the second image is captured less than about 1, 0.5, 0.1,or 0.05 seconds after the first image is captured.

At 440, an inference is made as to whether an object is touching displayscreen 160. For example, after the object of interest is located, aregion surrounding the location may be analyzed in the second image toassess any shadow surrounding the object. The inference may be madebased at least partly, primarily or completely on the second image. Forexample, if an object (e.g., a fingertip) is not surrounded by a shadow,it may be inferred that the object is touching display screen 160. Insome instances, the inference involves assessing a binary query, suchas: is the object touching screen 160 or not? In other instances, thequery involves more than two possible outcomes, such as: is the objectfar from screen 160, near screen 160 or touching screen 160, or whatportion of the object is touching screen 160 (e.g., determined based ona portion of the object surrounded by little to no shadow)? In someembodiments, it may be determined whether the object is hovering nearthe screen, for example within a threshold distance of the screen and/orfor a threshold amount of time.

At 425, if the object is inferred to be touching the screen, interactionbetween the projected image and the object may be allowed. For example,a user may be able to select programs, progress through displays,highlight portions of the displayed screen, provide input into a programusing a displayed user interface, etc. In some instances, the object(e.g., a user's fingertip) may act substantially as a computer mousewhen the object is inferred to be touching the screen. In someinstances, display screen 160 acts substantially as a track pad when theobject is inferred to be touching the screen. For example, an estimationof an object's location and an inference of screen contact may allow foran inference as to whether the object is touching the screen, tappingthe screen, dragging along the screen, and/or being removed from thescreen. A program may associate object actions with specific commands(e.g., to select a program, change views, etc.).

Any portion of the method depicted in FIG. 4 may be repeated. Forexample, while a single or set of reference images are projected, firstand second regions may be repeatedly illuminated (e.g., during distincttime periods). Images may be repeatedly captured. An estimation as to anobject's location and an inference as to whether it is touching screen160 may also be repeatedly made. Arrows indicating repeated actions inFIG. 4 are for example only. It is understood that the method mayinclude a different repetition. Further, it will be understood that,e.g., process 400 can include the performance of blocks 410-420 beforeperformance of blocks 430-440 or the reverse.

FIG. 5 shows an embodiment of a method 500 for estimating a location ofan object (e.g., as in 430 of FIG. 4). At 505, one or more backgroundimages are obtained. A background image may be substantially similar tothe first image and/or to the second image (e.g., the first imagecaptured in 415 of FIG. 4), except that no object is in the backgroundimage. For example, the background image(s) may be captured using a samecamera/s, having same settings, from a same perspective, and/or withsubstantially similar illumination as present during capture of thefirst/second image. In one instance, a first background image iscaptured with conditions (e.g., settings, perspective, illumination,etc.) similar to those used or those that will likely be used to capturethe first image. A second background image may be captured withconditions (e.g., settings, perspective, illumination, etc.) similar tothose used or that will be used to capture the second image.

The background image(s) may have been captured during a controlcondition when no object was present. The background image(s) may havebeen specifically captured to act as a control (e.g., after havinginstructed a user to prepare a scene for the capture of the image) or itmay have been taken without a user's knowledge. For example, projectionsystem 100 may repeatedly image a scene from a single perspective. Atechnique, such as determining brightness variability or assessingwhether there is any above-threshold difference (e.g., a cumulative sumof absolute differences in pixel-paired intensities, the sum beingacross pixels) across successive images, may be used to identify animage in which it is likely that no object is in front of screen 160. Itwill be appreciated that the terms brightness, intensity and luminanceare used interchangeably herein when used with respect to analysis of animage.

At 510, the first and/or second image is filtered based on the (e.g.,respective) background image(s). In one instance, a first image isfiltered based on a corresponding first background image, and a secondimage is filtered based on a corresponding second background image. Forexample, a two-dimensional brightness map from the background image(s)(B_(x,y)) may be subtracted from a similar map of the first and/orsecond image (I_(x,y)). A noise threshold, e, may be set, such that thefiltered image only includes non-zero values at pixels for which adifference between the first and/or second image and the backgroundimage exceeded the threshold. This process may be represented asfollows:

$j_{x,y} = \left\{ \begin{matrix}{I_{x,y} - B_{x,y}} & {{{if}\mspace{14mu}{{I_{x,y} - B_{x,y}}}} > ɛ} \\0 & {otherwise}\end{matrix} \right.$Additional or alternative filtering techniques may be applied. Forexample, a first, second and/or background image may be scaled by ascalar or adjustable scalar. For example:

$B_{f} = \left\{ {\left. \frac{I_{x,y}}{v} \middle| {v \neq 0} \right.,{1 \leq x \leq {Width}},{1 \leq y \leq {Height}}} \right\}$$j_{f} = \left\{ \begin{matrix}{\frac{I_{({x,y})}}{v} - B_{f{({x,y})}}} & {{{if}\mspace{14mu}{{\frac{I_{({x,y})}}{v} - B_{f{({x,y})}}}}} > ɛ} \\0 & {otherwise}\end{matrix} \right.$Such scaling may be less advantageous or unnecessary if camera settingsmay be adjusted before capturing different images. The filtering at 510may produce a filtered image (j) that does not include static objects,such as a screen, a podium, etc.

At 515, a location associated with the object may be estimated, e.g.,based on the filtered image. Various computer-vision techniques (e.g.,edge detection or feature detection) may be used to identify theobject's location.

In some instances, a top non-zero pixel or the top pixel exceeding athreshold in the filtered image is identified. If multiple columnsinclude a pixel meeting the desired criteria in a same top row, a singletop pixel may be identified by, e.g., choosing a left-most, center orright-most column. In one embodiment, the top pixel is identified bysearching for a first non-zero or above-threshold pixel while scanningfrom a top-left corner of the image to the bottom-right corner. The toppixel may itself represent the estimated location or may serve as anintermediate variable used to estimate the location. For example, afterthe top pixel is identified, an extremum position within an area (e.g.,a rectangle) surrounding the top pixel may be identified. Thus, forexample, it may be determined that a top pixel has coordinates (15, 60).An extremum pixel within a rectangular block extending from (10,55) to(20,65) may then be identified as the estimated location.

FIG. 6a shows an embodiment of a method 600 a for determining whether anobject is touching the screen (e.g., as in 440 of FIG. 4). At 605, apoint of interest in an image (e.g., in second image captured in 425 ofFIG. 4) may be identified. The point of interest may include thelocation estimated to be associated with the object in FIG. 5. Forexample, the point of interest may comprise a pixel estimated tocorrespond to a top of the object (e.g., a top of a fingertip).

At 610, a region of interest may be identified. The region may includean area surrounding the point of interest. For example, the region mayinclude a rectangular region surrounding the point of interest.

At 615, brightness variation within the region is analyzed. When anobject is casting shadows (hovering) over the screen, the brightnessvariation may be large due to dramatic differences between a shadow'sdark brightness values and the object's brightness values. When theobject approaches the screen, most pixels may represent the object,causing relatively low brightness variations. When the object touchesthe screen, a bright border may form at the top of the object, therebyagain increasing the brightness variations.

The analysis may include computing a standard deviation or variance ofbrightness values within the region. In one embodiment, a plurality ofvariation measures (e.g., standard deviations or modified standarddeviations) are calculated—each associated with a different portion ofthe region (e.g., a different column of pixels). In one embodiment, thevariation measures comprise modified standard deviations, which are notdivided by an n-value (number of elements); this modification may reducethe measurements' susceptibility to varying block sizes. A singlecombined variation measure may be calculated based on the plurality ofvariation measures (e.g., equal to a median or mean of the plurality ofvariation measures). The combined variation measure may be scaled, e.g.,to account for a size of the analyzed region.

A mathematical representation is shown below. In this embodiment, afiltered image is calculated by subtracting a background image's (e.g.,brightness) values from, e.g., a second image (described above). Thefiltered image is set to zero if the difference is below a noisethreshold, ε. A top block, K, surrounding a location of interest, (i,j)is analyzed. Modified standard deviations are computed for each column,σ_(i). The modified standard deviations are then averaged to compute atime-dependent combined variation measure, a_(t).

Δ = {δ_(x, y)|1 ≤ x ≤ width, 1 ≤ y ≤ height}$\delta_{x,y} = \left\{ {{\begin{matrix}{I_{x,y} - B_{x,y}} & {{{if}\mspace{14mu}{{I_{x,y} - B_{x,y}}}} \geq ɛ} \\0 & {otherwise}\end{matrix}K} = {{{{TopBlock}\left( \Delta_{t} \right)}{where}K} = {{\left\{ {\left. k_{i,j} \middle| {{x_{k} - \frac{l_{1}}{2}} \leq i \leq {x_{k} + \frac{l_{1}}{2}}} \right.,{{y_{k} - \frac{l_{2}}{2}} \leq j \leq {y_{k} + \frac{l_{2}}{2}}}} \right\}\sigma_{i}} = {{\sqrt{N \cdot {E\left\lbrack \left( {K_{i} - \mu_{i}} \right)^{2} \right\rbrack}}{where}K_{i}} = {{\left\{ {{\left. k_{i,j} \middle| i \right. = c},{{y_{k} - \frac{l_{2}}{2}} \leq j \leq {y_{k} + \frac{l_{2}}{2}}}} \right\} a_{t}} = {\frac{1}{l_{1}}{\sum\limits_{1}^{l_{2}}\;\sigma_{i}}}}}}}} \right.$

The average standard deviation may be used, e.g., to determine whetheran object is approaching a screen. Specifically, a low average standarddeviation can be used to infer that an object is approaching the screenand not hovering over the screen (in which case the object would besurrounded by a shadow) or contacting the screen (in which case theobject would be surrounded by a bright spot). However, the averagestandard deviation may not distinguish between these two latterinstances in all embodiments.

At 620, a dark/light sign is identified. The sign may indicate, e.g.,whether an extremum point (or an average of a top n number of pointswith regard to intensity) within the region and/or the point of interestis associated with a “dark” intensity value or a “light’ intensityvalue. For example, a darkness sign may be identified when a brightnessof the point of interest is below a threshold or below an averagebrightness value. A light sign may be identified when a brightness ofthe point of interest is above a threshold or above an averagebrightness value. As another example, a skew of intensity distributionscan be used to determine the sign.

A threshold can be determined based on a surface on which an image isbeing displayed, ambient light, skin color, a color of a control object,or an image or display being projected. A threshold can be determined bydetermining a distribution of intensity values and setting the thresholdto a value for which a given number of the intensity values are below orabove. For example, a threshold for determining a dark sign can bedetermined by accessing a distribution of intensity values for an imagebeing projected, and setting the threshold such that 20% of the accessedvalues fall below the threshold. A threshold may be fixed or variable.For example, a threshold could vary depending on the image beingprojected or an average off-screen instantaneous intensity.

At 625, a determination is made as to whether the object is touching thescreen. The table below indicates variation measures and dark/lightsigns that are associated with various states. Thus, based on thevariation measure determined at 615 and the dark/light sign determinedat 620, a state of an object (e.g., hovering, approaching the screen, ortouching the screen) may be determined. FIG. 7 shows an example of astate machine that can track changes in situations when an objectapproaches the screen.

State Variation Measure Dark/Light Sign Hover High Dark ApproachingScreen Low n/a Touching Screen High Light

FIG. 6b shows an embodiment of another method 600 b for determiningwhether an object is touching the screen (e.g., as in 440 of FIG. 4). At655, a display is projected onto a surface. Block 655 can includeembodiments similar to embodiments described with respect to block 405.For example, the display can be projected by a projector onto a displayscreen and can be a digital image (e.g., a presentation slide, adocument, a desktop, a user interface, a picture, an animation, etc.).

At 660, a region near the surface is illuminated. Block 660 can includeembodiments similar to embodiments described with respect to block 430.For example, the region can comprise or consist of a front surface ofthe surface and/or a region just in front of the surface, and theillumination can be configured such that a user's hand will likely beilluminated if he stands between a projector projecting the display andthe surface and touches surface. The illumination may compriseillumination by non-visible light. This second-region illumination maybe in addition to light emitted to project the reference image in 400(which would include emission of visible light). Thus, the region nearthe surface may be illuminated at 660 while the display is beingprojected onto the surface in 655.

At 665, an image of the surface is captured during illumination. Block665 can include embodiments similar to embodiments described withrespect to block 435. At 670, variation of intensities of pixels in atleast a portion of the image is analyzed. Block 670 can includeembodiments similar to embodiments described with respect to block 615.For example, the analysis may include computing one or more (traditionalor modified) standard deviation or variance of brightness values withinthe region.

At 675, a polarity associated with the portion is determined. Block 675can include embodiments similar to embodiments described with respect toblock 620. For example, determining the polarity can include identifyinga sign of an extremum in the region, identifying a direction of a skewof a distribution of intensity values from the region or comparing anextremum in the region to a threshold.

At 680, a determination is made as to whether the object is touching thesurface based at least in part on the analysis of the variation ofintensities of the pixels and on the determined polarity. Block 680 caninclude embodiments similar to embodiments described with respect toblock 625. For example, it may be determined that the object is touchingthe surface when the portion includes a high variation in pixelintensities and a polarity is high or light. In some embodiments, thedetermination of whether the object is touching the surface may affectoperation of an application, for example an application associated withthe display being projected in 655.

FIG. 8 shows one embodiment of a projection system 800. The componentsshown in FIG. 8 may be parts of a single or multiple devices. Forexample, some components may be parts of a projector, while othercomponents may be parts of an independent (coupled or non-coupled)computer. It is understood that, in some embodiments, one or moredepicted components may be omitted from the system and/or additionalcomponents may be added to the depicted system.

System 800 includes a camera 820 (e.g., camera 120), one or more lightsources 812 (e.g., front light source 100 and/or back light source 115),and timing circuit 825 (e.g., timing circuit 125). Camera 820 maycapture images of a region at or near a display screen. One or morelight sources 812 may illuminate a region at or near the screen. Lightsource(s) 812 may emit non-visible (e.g., infrared) light. Camera 820and light source(s) 812 may be coupled, such that images are capturedwhile light is emitted. In one embodiment, camera 820 is coupled tolight source (s) 812 via timing circuit 825. In embodiments in whichlight source(s) 812 includes a plurality of lights, timing circuit 825may also or alternatively coordinate the emission of these lights (e.g.,such that they emit light during substantially non-overlapping timeperiods).

Images captured by camera 820 may be processed and/or analyzed. In oneembodiment, images are input to image pre-processor 870. Imagepre-processor 870 may include, e.g., a filter 872. Filter 872 may filterout particular spatial frequencies, enhance the image, crop the image,add or subtract an intensity value (e.g., a mean intensity) to allpixels, etc. In some instances, filter 872 includes a static-objectremover 874. Static-object remover 874 may filter out parts of thecaptured image not of interest (e.g., inanimate objects, the screen,objects that have not moved within a given time period, etc.). In someinstances, static-object remover 874 uses a control image to performthis filtering. For example, an image comprising only the screen or withno people in the image may be subtracted from subsequent images. Asanother example, the control image comprises an average of multipleimages (e.g., such that only static objects remain pronounced in thecontrol image). Pre-processor 870 may include a de-noiser 876. De-noiser876 may, for example, identify pixels with values (e.g., brightnessvalues) insufficiently different from corresponding values in a controlimage. Values for these pixels may then be set to zero. Pre-processor870 may include a normalizer 878. For example, normalizer 878 mayattempt to control for environmental lighting. Thus, for example, it maybe predicted that particular pixels will always be unobstructed. Afiltered image may then be adjusted such that pixels at these values areequal to identified values or within identified ranges.

Pre-processed images may be analyzed by one or more image analyzer(s)880. In some embodiments, the analysis performed by image analyzer 880depends on characteristics associated with an image at issue. Forexample, analysis of an image obtained while a front light source wasemitting light may differ from analysis of an image obtained while asecond light source was emitting light. Image analyzer 880 may includean edge detector 882. Edge detector 882 may, e.g., identify lines and/orpixel variations to identify edges. The edges may be edges associatedwith an object (e.g., a fingertip), an object's shadow and/or abrightness border around the object.

Image analyzer 880 may include a feature recognizer 884. Featurerecognizer 884 may recognize features in general or particular types offeatures. For example, feature recognizer 884 may identify that a sceneincludes two features (e.g., which may correspond to an object and ashadow). In some embodiments, feature recognizer 884 may particularlyrecognize whether a scene includes an object, such as a fingertip.Feature recognizer 884 may analyze an image's brightness, brightnessvariation, brightness correlations, etc.

Image analyzer 880 may include variation detector 886. Variationdetector 886 may identify, e.g., a variation, range or standarddeviation of pixel values (e.g., pixel brightness values). Variationdetector 886 may identify a variable output as an absolute or scaledvalue (e.g., a percentage). Variation detector 886 may analyze pixelsacross an entire image or within a portion of the image (e.g.,surrounding a location associated with an object).

Image analyzer 880 may include sign detector 888. Sign detector 888 mayanalyze whether the image includes a dark feature (e.g., a fingertip'sshadow) or a bright feature (e.g., a bright spot on top of a fingertiptouching a screen). Sign detector 888 may analyze an entire image or aportion of the image (e.g., surrounding a location associated with anobject). Sign detector may identify a sign of one or more extremumpixels or the sign of a median, mean or mode pixel in the region.

Image analyzer 880 may be coupled to an object detector 892, an objectlocator 894 and/or an object state detector 896. Object detector 892 maydetect whether an object (in general or a specific object, such as afingertip) is present in an image. Object locator 894 may estimate alocation of an object. Object state detector 896 may infer a state of anobject (e.g., hovering, idle, touching a screen, or approaching ascreen). Object detector 892, object locator 894 and/or object statedetector 896 may be coupled to program interactor 898. For example, auser may be able to interact with a program only when an object isdetected and is detected as being in a particular state (e.g., touchinga screen). A type of interaction may depend on a location of the object(e.g., such that a program responds differently depending on a locationof a user's hand on a screen). Program interactor 898 may be coupled toprojection optics 805 (e.g., projection optics 105), such that, e.g.,what is displayed by the optics depends on an object's presence,location and/or state. Thus, projection optics 805 or other suchprojection elements may be used to project images or otherrepresentations of a state or operation of a program or application orcomputer system. The user may interact with the program, application, orcomputer system by touching a surface on which the images orrepresentations are projected or displayed. For example, when system 800detects that a user touches a surface on which an image is displayed,the system may treat the touch as a clicking event (similar to how auser may use a mouse). The user may therefore select objects displayedon the surface by touching them. Further the user may touch the surfaceand drag his finger across the surface to create a selection box, or theuser may touch an object and drag his finger across the surface to movethe object, for example. Those of skill in the art will appreciate thatthe surface may therefore be operated as a touch-screen device, evenwhen the system 800 is remote from the surface and the surface comprisesno electronic components. In this way, a determination of whether a useror other object is touching the surface and/or a location of the objector touch may be provided and/or used as an input to an application orcomputer process, for example based on an image or other display beingprojected. In some embodiments, the application may be running on orimplemented by the system 100 or 800. In some embodiments, theapplication may be running on a computer coupled to the system 100 or800, and input may be provided to the computer system via acommunication interface.

Projection system 100 or 800 may include and/or be coupled to a computersystem. For example, projection system 100 may be coupled to a computersystem that controls what images are to be displayed on display screen160. An internal or external computer may also be used to determine,e.g., a location of an object and/or whether the object is touching thescreen. FIG. 9 provides a schematic illustration of one embodiment of acomputer system 900 that can, e.g., perform all or part of methodsdescribed herein the methods described herein. Any or all of computersystem 900 may be incorporated in projection system 100 and/or coupled(e.g., through a wireless or wired connection) to projection system 100.FIG. 9 is meant only to provide a generalized illustration of variouscomponents, any or all of which may be utilized as appropriate. FIG. 9,therefore, broadly illustrates how individual system elements may beimplemented in a relatively separated or relatively more integratedmanner.

The computer system 900 is shown comprising hardware elements that canbe electrically coupled via a bus 905 (or may otherwise be incommunication, as appropriate). The hardware elements may include one ormore processors 910, including without limitation one or moregeneral-purpose processors and/or one or more special-purpose processors(such as digital signal processing chips, graphics accelerationprocessors, and/or the like); one or more input devices 915, which caninclude without limitation a mouse, a keyboard and/or the like; and oneor more output devices 920, which can include without limitation adisplay device, a printer and/or the like.

The computer system 900 may further include (and/or be in communicationwith) one or more storage devices 925, which can comprise, withoutlimitation, local and/or network accessible storage, and/or can include,without limitation, a disk drive, a drive array, an optical storagedevice, solid-state storage device such as a random access memory(“RAM”) and/or a read-only memory (“ROM”), which can be programmable,flash-updateable and/or the like. Such storage devices may be configuredto implement any appropriate data stores, including without limitation,various file systems, database structures, and/or the like.

The computer system 900 might also include a communications subsystem930, which can include without limitation a modem, a network card(wireless or wired), an infrared communication device, a wirelesscommunication device and/or chipset (such as a Bluetooth™ device, an802.11 device, a WiFi device, a WiMax device, cellular communicationfacilities, etc.), and/or the like. The communications subsystem 530 maypermit data to be exchanged with a network (such as the networkdescribed below, to name one example), other computer systems, and/orany other devices described herein. In many embodiments, the computersystem 900 will further comprise a working memory 935, which can includea RAM or ROM device, as described above.

The computer system 900 also can comprise software elements, shown asbeing currently located within the working memory 935, including anoperating system 940, device drivers, executable libraries, and/or othercode, such as one or more application programs 945, which may comprisecomputer programs provided by various embodiments, and/or may bedesigned to implement methods, and/or configure systems, provided byother embodiments, as described herein. Merely by way of example, one ormore procedures described with respect to the method(s) discussed abovemight be implemented as code and/or instructions executable by acomputer (and/or a processor within a computer); in an aspect, then,such code and/or instructions can be used to configure and/or adapt ageneral purpose computer (or other device) to perform one or moreoperations in accordance with the described methods.

A set of these instructions and/or code might be stored on acomputer-readable storage medium, such as the storage device(s) 925described above. In some cases, the storage medium might be incorporatedwithin a computer system, such as the system 900. In other embodiments,the storage medium might be separate from a computer system (e.g., aremovable medium, such as a compact disc), and/or provided in aninstallation package, such that the storage medium can be used toprogram, configure and/or adapt a general purpose computer with theinstructions/code stored thereon. These instructions might take the formof executable code, which is executable by the computer system 900and/or might take the form of source and/or installable code, which,upon compilation and/or installation on the computer system 900 (e.g.,using any of a variety of generally available compilers, installationprograms, compression/decompression utilities, etc.) then takes the formof executable code. In some embodiments, system 900 or componentsthereof are included within a housing of system 100 or 800, or mayotherwise be mechanically coupled thereto. For example, the one or moreprocessors 910 may comprise one or more of image pre-processor 870,image analyzer 880, object detector 892, object locator 894, objectstate detector 896 or program interactor 898.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware might also be used, and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

As mentioned above, in one aspect, some embodiments may employ acomputer system (such as the computer system 900) to perform methods inaccordance with various embodiments of the invention. According to a setof embodiments, some or all of the procedures of such methods areperformed by the computer system 900 in response to processor 910executing one or more sequences of one or more instructions (which mightbe incorporated into the operating system 940 and/or other code, such asan application program 945) contained in the working memory 535. Suchinstructions may be read into the working memory 935 from anothercomputer-readable medium, such as one or more of the storage device(s)925. Merely by way of example, execution of the sequences ofinstructions contained in the working memory 935 might cause theprocessor(s) 910 to perform one or more procedures of the methodsdescribed herein. For example, one or more processors 910 can performblocks 420, 425, and/or 440 of process 400; blocks 505, 510 and/or 515of process 500; blocks 605, 610, 615, 620 and/or 625 of process 600 aand/or blocks 670, 675, and/or 680 of process 600 b. Further, the one ormore processors 910 may instruct and/or cause one or more otherelements, such as the projector 105, the light source 110, the lightsource 115, and/or the camera 120, for example, to perform blocks 405,410, 415, 430, and/or 435 of process 400; and/or blocks 655, 660, and/or665 process 600 b.

The terms “machine-readable medium” and “computer-readable medium,” asused herein, refer to any medium that participates in providing datathat causes a machine to operate in a specific fashion. Computerreadable medium and storage medium do not refer to transitorypropagating signals. In an embodiment implemented using the computersystem 900, various computer-readable media might be involved inproviding instructions/code to processor(s) 910 for execution and/ormight be used to store such instructions/code. In many implementations,a computer-readable medium is a physical and/or tangible storage medium.Such a medium may take the form of a non-volatile media or volatilemedia. Non-volatile media include, for example, optical and/or magneticdisks, such as the storage device(s) 925. Volatile media include,without limitation, dynamic memory, such as the working memory 935.

Common forms of physical and/or tangible computer-readable mediainclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, or any other magnetic medium, a CD-ROM, any other opticalmedium, punchcards, papertape, any other physical medium with patternsof holes, a RAM, a PROM, EPROM, a FLASH-EPROM, any other memory chip orcartridge, etc.

Examples

FIG. 10a shows an example of a situation in which a “second image” iscaptured as described in relation to 425 of FIG. 4. Specifically, a backlight source is emitting light while the image is captured. In thisinstance, a hand is hovering over the screen. Due to the angle andintensity of the illumination, a shadow is cast under the hand. FIGS.10b and 10c show two perspectives of a background-subtracted brightnessmap calculated based on the second image. The z-axis of the map and thecoloring indicate the brightness of each pixel. Positive values indicatethat the pixel is brighter than the background, and negative valuesindicate that the pixel is darker than the background. As evidenced(e.g. particularly from the side-view perspective in FIG. 10c ), thetop-most pronounced values are negative—representing a shadow.

FIG. 11a shows another example of a situation in which a “second image”is captured as described in relation to 425 of FIG. 4. In this instance,a finger is touching the screen. As shown, the finger is surrounded by abright region. FIGS. 11b and 11c show two perspectives of abackground-subtracted brightness map calculated based on the secondimage. The z-axis of the map and the coloring again indicate thebrightness of each pixel. As evidenced (e.g., particularly from theside-view perspective in FIG. 11c ), the top-most pronounced values arepositive—representing a bright contact border. Thus, the second imagesmay be used to distinguish situations in which an object is hoveringover a screen from situations in which an object is touching a screen.

The methods, systems, and devices discussed above are examples. Variousconfigurations may omit, substitute, or add various procedures orcomponents as appropriate. For instance, in alternative configurations,the methods may be performed in an order different from that described,and/or various stages may be added, omitted, and/or combined. Also,features described with respect to certain configurations may becombined in various other configurations. Different aspects and elementsof the configurations may be combined in a similar manner. Also,technology evolves and, thus, many of the elements are examples and donot limit the scope of the disclosure or claims.

Specific details are given in the description to provide a thoroughunderstanding of example configurations (including implementations).However, configurations may be practiced without these specific details.For example, well-known circuits, processes, algorithms, structures, andtechniques have been shown without unnecessary detail in order to avoidobscuring the configurations. This description provides exampleconfigurations only, and does not limit the scope, applicability, orconfigurations of the claims. Rather, the preceding description of theconfigurations will provide those skilled in the art with an enablingdescription for implementing described techniques. Various changes maybe made in the function and arrangement of elements without departingfrom the spirit or scope of the disclosure.

Also, configurations may be described as a process which is depicted asa flow diagram or block diagram. Although each may describe theoperations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be rearranged. A process may have additional steps notincluded in the figure. Furthermore, examples of the methods may beimplemented by hardware, software, firmware, middleware, microcode,hardware description languages, or any combination thereof. Whenimplemented in software, firmware, middleware, or microcode, the programcode or code segments to perform the necessary tasks may be stored in anon-transitory computer-readable medium such as a storage medium.Processors may perform the described tasks.

Having described several example configurations, various modifications,alternative constructions, and equivalents may be used without departingfrom the spirit of the disclosure. For example, the above elements maybe components of a larger system, wherein other rules may takeprecedence over or otherwise modify the application of the invention.Also, a number of steps may be undertaken before, during, or after theabove elements are considered. Accordingly, the above description doesnot bound the scope of the claims.

What is claimed is:
 1. A projector system, comprising: projection opticsconfigured to project a display on a surface from a front side of thesurface; a first light source configured to emit light in a firstdirection towards the surface from the front side of the surface; asecond light source configured to emit light in a second directiontowards the surface from the front side of the surface, wherein thelight emitted in the second direction is emitted through the projectionoptics via a prism positioned between the second light source and a lensof the projection optics; and an image analyzer configured to: determinea position of an object placed in front of the surface, based at leastin part on the emitted light in the first direction, wherein theposition of the object is determined based on detecting one or moreedges of the object in at least one image captured from the front sideof the surface; and determine whether the object is touching the surfacebased at least in part on the emitted light in the second direction. 2.The projector system of claim 1, wherein the first light source and thesecond light source are configured to emit light in an alternatingfashion.
 3. The projector system of claim 1, wherein the projectorsystem is a short-throw projector.
 4. The projector system of claim 1,wherein determining the position of the object comprises determining oneor more pixel coordinates associated with the object.
 5. The projectorsystem of claim 1, wherein the image analyzer is further configured todetermine a region of interest that includes the determined position ofthe object, and wherein determining whether the object is touching thesurface is based at least in part on the determined region of interest.6. The projector system of claim 5, wherein the image analyzer isfurther configured to analyze brightness variation within the region ofinterest, and wherein determining whether the object is touching thesurface is based at least in part on the analyzed brightness variation.7. A method, comprising: projecting a display on a surface from a frontside of the surface; emitting, via a first light source, light in afirst direction towards the surface from the front side of the surface;emitting, via a second light source, light in a second direction towardsthe surface from the front side of the surface, wherein the lightemitted in the second direction is emitted through the projection opticsvia a prism positioned between the second light source and a lens of theprojection optics; determining a position of an object placed in frontof the surface, based at least in part on the emitted light in the firstdirection, wherein the position of the object is determined based ondetecting one or more edges of the object in at least one image capturedfrom the front side of the surface; and determining whether the objectis touching the surface based at least in part on the emitted light inthe second direction.
 8. The method of claim 7, wherein the lightemitted in the first direction and the light emitted in the seconddirection are emitted in an alternating fashion.
 9. The method of claim7, wherein the display is projected on the surface via a short-throwprojector.
 10. The method of claim 7, wherein determining the positionof the object comprises determining one or more pixel coordinatesassociated with the object.
 11. The method of claim 7, furthercomprising determining a region of interest that includes the determinedposition of the object, and wherein determining whether the object istouching the surface is based at least in part on the determined regionof interest.
 12. The method of claim 11, wherein further comprisinganalyzing brightness variation within the region of interest, andwherein determining whether the object is touching the surface is basedat least in part on the analyzed brightness variation.
 13. A projectorsystem, comprising: means for projecting a display on a surface from afront side of the surface; means for emitting, via a first light source,light in a first direction towards the surface from the front side ofthe surface; means for emitting, via a second light source, light in asecond direction towards the surface from the front side of the surface,wherein the light emitted in the second direction is emitted through theprojection optics via a prism positioned between the second light sourceand a lens of the projection optics; means for determining a position ofan object placed in front of the surface, based at least in part on theemitted light in the first direction, wherein the position of the objectis determined based on detecting one or more edges of the object in atleast one image captured from the front side of the surface; and meansfor determining whether the object is touching the surface based atleast in part on the emitted light in the second direction.
 14. Theprojector system of claim 13, wherein the light emitted in the firstdirection and the light emitted in the second direction are emitted inan alternating fashion.
 15. The projector system of claim 13, whereinthe display is projected on the surface via a short-throw projector. 16.The projector system of claim 13, wherein the means for determining theposition of the object comprises means for determining one or more pixelcoordinates associated with the object.
 17. The projector system ofclaim 13, further comprising means for determining a region of interestthat includes the determined position of the object, and whereindetermining whether the object is touching the surface is based at leastin part on the determined region of interest.
 18. The projector systemof claim 17, further comprising means for analyzing brightness variationwithin the region of interest, and wherein determining whether theobject is touching the surface is based at least in part on the analyzedbrightness variation.
 19. A non-transitory computer-readable mediumcontaining instructions which, when executed by a processor, cause theprocessor to perform steps of, comprising: projecting a display on asurface from a front side of the surface; emitting, via a first lightsource, light in a first direction towards the surface from the frontside of the surface; emitting, via a second light source, light in asecond direction towards the surface from the front side of the surface,wherein the light emitted in the second direction is emitted through theprojection optics via a prism positioned between the second light sourceand a lens of the projection optics; determining a position of an objectplaced in front of the surface, based at least in part on the emittedlight in the first direction, wherein the position of the object isdetermined based on detecting one or more edges of the object in atleast one image captured from the front side of the surface; anddetermining whether the object is touching the surface based at least inpart on the emitted light in the second direction.
 20. Thenon-transitory computer-readable medium of claim 19, wherein the lightemitted in the first direction and the light emitted in the seconddirection are emitted in an alternating fashion.
 21. The non-transitorycomputer-readable medium of claim 19, wherein the display is projectedon the surface via a short-throw projector.
 22. The non-transitorycomputer-readable medium of claim 19, wherein determining the positionof the object comprises determining one or more pixel coordinatesassociated with the object.
 23. The non-transitory computer-readablemedium of claim 19, wherein the instructions further cause the processorto determine a region of interest that includes the determined positionof the object, and wherein determining whether the object is touchingthe surface is based at least in part on the determined region ofinterest.
 24. The non-transitory computer-readable medium of claim 23,wherein the instructions further cause the processor to analyzebrightness variation within the region of interest, and whereindetermining whether the object is touching the surface is based at leastin part on the analyzed brightness variation.